Skip to content

feat: limit in-flight operations in map/parallel#533

Draft
yaythomas wants to merge 1 commit into
mainfrom
feat/in-flight-max-concurrency
Draft

feat: limit in-flight operations in map/parallel#533
yaythomas wants to merge 1 commit into
mainfrom
feat/in-flight-max-concurrency

Conversation

@yaythomas

@yaythomas yaythomas commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Description of changes:
max_concurrency previously sized the thread pool only, so branches that suspended (invoke, wait, callbacks) released their thread and the next item started immediately. All items were initiated at once regardless of the configured limit.

Rework the concurrency executor around a single coordinator loop:

  • The calling thread owns all branch state and schedules work; worker threads only run branches and report outcomes on a queue. No locks remain in the module.
  • max_concurrency now bounds in-flight branches. A suspended branch keeps its slot until it reaches a terminal state, matching the JS and Java SDKs. When every in-flight branch is suspended, the parent suspends with the earliest resume timestamp, or indefinitely.
  • Branches start in index order and operation ids derive from the branch index, so scheduling stays deterministic across invocations.
  • Timed suspends resume in-process via the event queue timeout, replacing the TimerScheduler background thread.
  • Consolidate completion logic (thresholds and reason inference) into CompletionPolicy, replacing ExecutionCounters and the duplicated logic in BatchResult.
  • Stop scheduling when a branch is orphaned: an ancestor completed, so all further checkpoints under it would be rejected.
  • Record the completion decision in map/parallel summaries and obey it on replay: summaries carry completionReason plus the started-branch index set (startedIndexes, or completedIndexes when smaller), and replay reconstructs the exact live result — same items, same statuses, same reason — instead of re-deriving from child checkpoints. Summaries without the record (checkpoints written by earlier SDK versions, custom summary generators) fall back to checkpoint derivation, preserving old behavior. This also makes FLAT-nesting results reconstructable on replay, which checkpoint derivation cannot discriminate. Handlers now default the summary generator per-field, so user-provided configs get exact replay without opting in.

Behavior changes

  • BatchResult omits never-started branches (matching the JS SDK). Previously every item appeared as STARTED even when an early completion meant it never ran; total_count now reflects branches that actually started.
  • Configs with only min_successful set no longer fail fast on the first failure. The old tolerance check ignored min_successful when deciding whether criteria existed, so a single failure stopped scheduling; scheduling now continues past failures until min_successful is reached or all items finish, matching the JS and Java SDKs.
  • Config bounds are validated at construction, matching the Java SDK: max_concurrency and min_successful must be at least 1, tolerated_failure_count non-negative, tolerated_failure_percentage within 0–100. Previously max_concurrency=0 silently meant unlimited and min_successful=0 silently meant all items.

Testing

  • Unit: 1384 tests, typecheck and format clean, examples suite green.
  • Cloud conformance (aws-durable-execution-sdk-testing): map 20/20 (including 9-16 MapLargeResult and 9-20 MapOpSerdeReplay, which exercise the summarized-replay path), parallel 22/22 — including 9 previously failing requirements covering max_concurrency semantics and a success_count undercount when min_successful was not reached.

Issue #, if available:
Closes #279

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@yaythomas yaythomas added this to the v2 milestone Jul 14, 2026
@yaythomas yaythomas marked this pull request as draft July 14, 2026 18:28
@yaythomas yaythomas force-pushed the feat/in-flight-max-concurrency branch 4 times, most recently from 9f04c8d to 3aa2bdc Compare July 16, 2026 17:26
@ayushiahjolia ayushiahjolia removed this from the v2 milestone Jul 16, 2026
@yaythomas yaythomas force-pushed the feat/in-flight-max-concurrency branch from 3aa2bdc to 0940152 Compare July 16, 2026 18:42
max_concurrency previously sized the thread pool only, so branches
that suspended (invoke, wait, callbacks) released their thread and
the next item started immediately. All items were initiated at once
regardless of the configured limit.

Rework the concurrency executor around a single coordinator loop:

- The calling thread owns all branch state and schedules work; worker
  threads only run branches and report outcomes on a queue. No locks
  remain in the module.
- max_concurrency now bounds in-flight branches. A suspended branch
  keeps its slot until it reaches a terminal state, matching the JS
  and Java SDKs. When every in-flight branch is suspended, the parent
  suspends with the earliest resume timestamp, or indefinitely.
- Branches start in index order and operation ids derive from the
  branch index, so scheduling stays deterministic across invocations.
- Timed suspends resume in-process via the event queue timeout,
  replacing the TimerScheduler background thread.
- Consolidate completion logic (thresholds and reason inference) into
  CompletionPolicy, replacing ExecutionCounters and the duplicated
  logic in BatchResult.
- Omit never-started branches from BatchResult, matching the JS SDK.
- Stop scheduling when a branch is orphaned: an ancestor completed,
  so all further checkpoints under it would be rejected.
- Record the completion decision in map/parallel summaries and obey
  it on replay: summaries carry the started-branch index set (or its
  complement, whichever is smaller) and replay reconstructs the exact
  live result instead of re-deriving statuses from child checkpoints.
  Summaries without the record fall back to checkpoint derivation.
  This also reconstructs FLAT results on replay, which checkpoint
  derivation could not discriminate.
- Validate config bounds at construction, matching the Java SDK:
  max_concurrency and min_successful must be at least 1,
  tolerated_failure_count non-negative, tolerated_failure_percentage
  within 0-100. Previously max_concurrency=0 silently meant
  unlimited and min_successful=0 silently meant all items.

Closes #279
@yaythomas yaythomas force-pushed the feat/in-flight-max-concurrency branch from 0940152 to 69100b1 Compare July 16, 2026 19:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature]: max_concurrency to limit operations rather than threads

2 participants